Add support for zod validation error callback in schema stream #80
Conversation
|
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
RollForReflex
had a problem deploying
to
OPENAI
GitHub Actions
Failure
|
love it - I tried to compose these packages in a few different ways before they all ended up in the current form but the high level on how things are organized/responsibilities of each are this: schema stream: pure streaming json parser - the schema input allows us to stub the final object which enables you to start reading from the response asap - value here is that you can define what the expected output will be so as it streams through you have a stub that is readable testObject.nestedObject.someKey for instance will be in the structure from the start with default value for the type or a provided default ( https://github.com/hack-dance/island-ai/blob/main/public-packages/schemaStream/src/utils/streaming-json-parser.ts#L100 ) zod stream: given a zod schema pass through to schema stream for streaming json over the wire - we don't do the fullzod schema.parse since that throws but we safe parse and update the _meta object on the response to show u in realtime what paths in the object have completed and are/are not valid ( instructor: uses both under the hood and implements retry logic/actual schema validation/parsing and extends the openai sdk to make it as close as possible to what people are already using which means to adopt you only need to change an import ( https://github.com/instructor-ai/instructor-js ) |
RollForReflex
had a problem deploying
to
OPENAI
GitHub Actions
Failure
| @@ -233,6 +238,14 @@ export class SchemaStream { | |||
|
|
|||
| parser.onToken = this.handleToken.bind(this) | |||
| parser.onValue = () => void 0 | |||
| parser.onEnd = () => { | |||
There was a problem hiding this comment.
instead of passing a conditional callback that depends on another flag - would it be easier for us to just allow an onComplete here and just by default pass through the safeparse results always?
feels like making sure that isStrictSchmea is set on the instance AND havaing to pass the callback might be a bit confusing/unintuitive
vs say just adding
.parse({ onComplete })
and we conditional add the onEnd wiht a safeParse inside of it? We could also jsut pasas through the finala result and any other meta we have for free
const stream = parser.parse({
onComplete:({ isValid, errors = [], data }) => {
errors = zodError.errors
}
})
There was a problem hiding this comment.
yeah I dig that, that way you can do whatever you want
There was a problem hiding this comment.
love it, updated
RollForReflex
had a problem deploying
to
OPENAI
GitHub Actions
Failure
RollForReflex
had a problem deploying
to
OPENAI
GitHub Actions
Failure
RollForReflex
had a problem deploying
to
OPENAI
GitHub Actions
Failure
Was noodling on #78 a bit last night:
While we don't want to defacto throw validation errors (the point of the package after all is type-safety), we could provide a simple callback once the stream ends to say "hey, here's the errors we found" if the zod schema provided is strict.
This gives the caller the power to do whatever they want but they now have both the completed data via the stream and validation errors they may or may not care about.